Imaging apparatuses and enclosures

ABSTRACT

An enclosure having: a base face that is opaque or translucent to human eyes viewing from outside of the enclosure and transparent to infrared radiation; and at least two flat, orthogonal mounting faces configured to be overlaid respectively on at least two surfaces of walls and ceiling of a room. A thermal imaging apparatus configured to image based on infrared radiation and mounted within the enclosure with a predetermined orientation relative to the base face to have a designed imaging direction with respect to the room when the enclosure is mounted in the room to have the at least two orthogonal mounting faces overlaid respectively on the at least two surfaces.

RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 17/339,032, filed Jun. 4, 2021, issued as U.S. Pat.No. 11,533,413 on Dec. 20, 2020, which is a continuation application ofU.S. patent application Ser. No. 16/742,436, filed Jan. 14, 2020, issuedas U.S. Pat. No. 11,032,451 on Jun. 8, 2021, which is a continuationapplication of U.S. patent application Ser. No. 15/643,059, filed Jul.6, 2017, issued as U.S. Pat. No. 10,582,095 on Mar. 3, 2020, whichclaims the benefit of Prov. U.S. Pat. App. Ser. No. 62/408,615, filedOct. 14, 2016, all entitled “Imaging Apparatuses with Enclosures”, theentire disclosures of which applications are hereby incorporated hereinby reference.

The present application relates to U.S. patent application Ser. No.15/607,345, filed May 26, 2017 and entitled “Apparatus and Method ofLocation Determination in a Thermal Imaging System”, U.S. patentapplication Ser. No. 29/604,436, filed May 17, 2017 and entitled “CameraEnclosure”, U.S. patent application Ser. No. 14/750,403, filed Jun. 25,2015, published as U.S. Pat. App. Pub. No. 2015/0377711, and entitled“Apparatus and Method for Electromagnetic Radiation Sensing”, U.S.patent application Ser. No. 14/788,286, filed Jun. 30, 2015, andentitled “Micromechanical Device for Electromagnetic Radiation Sensing”,U.S. patent application Ser. No. 14/810,363, filed Jul. 27, 2015, andentitled “Micromechanical Device for Electromagnetic Radiation Sensing”,and U.S. patent application Ser. No. 15/188,116, filed Jun. 21, 2016,and entitled “Fabrication Method for Micromechanical Sensors”, theentire disclosure of which applications are hereby incorporated hereinby reference.

FIELD OF THE TECHNOLOGY

At least some embodiments disclosed herein relate, in general, toimaging apparatuses for monitoring, recording or imaging a spaceenclosed by walls and a floor (e.g., a room) to improve security,safety, energy usage, and to detect human occupancy and/or theiractivities, and more specific, to methods and apparatuses for setting upthe orientation of the imaging apparatuses (e.g., for thermal imaging).

BACKGROUND

A common method for monitoring a room for events and regions of interestis to mount within the room an imaging apparatus capable of detectingthe events or regions. The imaging apparatus may include a camera, astructured light source or similar apparatus, or some combinations ofsuch devices. Preferably, the imaging apparatus is mounted in a positionand orientation that allows it to detect events in a required region ofinterest, which may approach full room coverage.

A variety of methods, assemblies and apparatuses exist for the alignmentand installation of an imaging apparatus within a room. One method issimply to set up an imaging apparatus, such as an IP camera in a roomand adjust it manually with the feedback of a monitoring device tocapture a relevant region within a room, which has the disadvantage thatsome alignment tool, such as a display is required.

U.S. Pat. No. 6,912,007 discloses a securable corner mountedsurveillance unit with dual windows which is suited for a securedplacement in an upper corner of a room, including a closed circuitsurveillance camera unit. U.S. Pat. App. Pub. No. 2013/0155230 disclosesa method for the tilt and rotation of a camera to achieve desiredalignment. Such methods and apparatus have the disadvantage that theinstaller may be required to use some sort of elevation tool, such as aladder or stool to reach the designated area for the installation of theapparatus. Further some technical tools, such as screwdrivers, drills,etc. are required to secure the apparatuses accordingly and correctlyand to attach or install the power supply wires for the surveillancecamera. Manual or electric adjustments may be required for themonitoring device to capture the designated region.

U.S. Pat. No. 7,226,027 discloses a rear mounting member for receipt ofan insertion member, which requires an assembly of a shell, which againrequires for the installer to use some technical tools and to have theknowledge and skills to use such tools.

U.S. Pat. App. Pub. No. 2015/0377711, entitled “Apparatus and Method forElectromagnetic Radiation Sensing”, discloses an apparatus for thermalimaging based on infrared (IR) radiation. Such an apparatus can be usedfor human detection, fire detection, gas detection, temperaturemeasurements, environmental monitoring, energy saving, behavioranalysis, surveillance, information gathering and for human-machineinterfaces. Such an apparatus and/or other similar apparatuses can beused in embodiments of inventions disclosed in the present application.The entire disclosure of U.S. Pat. App. Pub. No. 2015/0377711 is herebyincorporated herein by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings in which like referencesindicate similar elements.

FIG. 1 illustrates an imaging apparatus having an enclosure adapted forsimplifying its installation in a room.

FIG. 2 shows an imaging apparatus assembly having an enclosure mountedon an edge of two orthogonal walls.

FIG. 3 shows a back top-down view of the enclosure illustrated in FIG. 2.

FIG. 4 shows an imaging apparatus assembly having an alternativeorientation mark.

FIG. 5 illustrates an imaging apparatus assembly with the base face ofthe enclosure being removed.

FIGS. 6 and 7 show an imaging apparatus assembly having a replaceablebattery unit on its bottom corner.

FIGS. 8 and 9 show imaging apparatus assemblies having alternativeshapes for their base faces.

FIG. 10 illustrate geometrical relations among the mounting position ofan enclosure, the orientation of the optical axis of an imagingapparatus housed within the enclosure, the field of view of the imagingapparatus, and the space within the room that can be captured in imagesobtained from the imaging apparatus housed within the enclosure.

FIG. 11 shows a thermal imaging system according to one embodiment.

FIG. 12 illustrates a method to measure mounting configurationparameters of a thermal imaging camera according to one embodiment.

FIG. 13 shows a user interface to obtain a user input to determine amounting height of a camera according to one embodiment.

FIG. 14 shows a data processing system that can be used to implementsome components of embodiments of the present application.

DETAILED DESCRIPTION

The following description and drawings are illustrative and are not tobe construed as limiting. Numerous specific details are described toprovide a thorough understanding. However, in certain instances, wellknown or conventional details are not described in order to avoidobscuring the description. References to one or an embodiment in thepresent disclosure are not necessarily references to the sameembodiment; and, such references mean at least one.

In one aspect, conventional methods to install an imaging apparatus in aroom to monitor a region of interest within the room exhibit one or moreof problems or drawbacks.

For example, the mounting of the imaging apparatus may be difficult toperform using existing methods, assemblies and apparatuses, which mayrequire a person with technical skills and tools to properly install theimaging apparatus in the room.

For example, a person with technical skills and tools may be required toachieve the mounting position and orientation necessary to cover therequired region of interest. For the alignment of the orientation of theimaging apparatus with respect to the region of interest, an installermay require a specialized monitoring device to display the imagerecorded by the imaging apparatus such that based on the display on thespecialized monitoring device, the installer adjusts the orientation ofthe imaging device to achieve the alignment.

Further, the installer may be required to have some technical skills inorder to align the imaging apparatus accordingly within the desiredorientation. For example, a slight change in the position andorientation of the imaging apparatus typically can result in asignificant change of the field of view of the imaging apparatus; and asa result, an installer may be required to have a steady hand and theability to adjust the position and/or orientation of an imagingapparatus finely, which can be challenging for seniors or people withdisabilities.

In the present disclosure, the imaging apparatus is configured to behoused in an enclosure that has mounting surfaces adapted to simplifythe installation of the imaging apparatus for proper alignment with thefield to be monitored by the imaging apparatus.

In another aspect, there are concerns associated with the installationof an imaging device in a room.

For example, observing an imaging apparatus mounted in a room may causea privacy intrusion concern among occupants who notice the imagingapparatus pointing towards them or the area where they are standing.

Being monitored is a concern and people are likely to behave differentlywhen they are aware that they are being recorded, monitored or imagedwith a visually noticeable imaging apparatus. Further, being monitoredwithin rooms may be restricted by privacy laws.

However, monitoring a room using, for example, a low-resolution thermalcamera will disallow the identification of individuals due to the lackof recorded and/or imaged detail. A low-resolution thermal camera orthermal sensor can detect humans by their elevated body temperature butlacks the spatial detail to make assumptions of the subject's identity.Such a low-resolution thermal camera may have the appearance of animaging device or lens; and the sight of such lens or imaging devices bya subject within the imaged region may be bothersome.

In the present disclosure, the imaging apparatus is configured to beoptionally hidden behind a visually non-transparent, but thermalradiation transparent material, which arrangement provides a solution toavoid privacy concerns of monitored subjects, while the subjects areonly monitored for their detection and not identification.

In other aspects, installation of an imaging apparatus in a room mayrequire a separate installation of cables for the power supply and/ordata transmission of the recorded footage. Wireless apparatusestypically have a very limited operational time due to the relativelylarge data volumes to be wirelessly transmitted from visual monitoringwith a low, standard or high definition camera. Typically, camerasrequire a resolution of above 320×240 pixels for good recognition of themonitored area and typically run at more than 1 frame per second.

The present disclosure provides solutions to simplify installation andalignment via an enclosure (167) of the imaging apparatus and hide theimaging apparatus within the enclosure (167), as illustrated in FIG. 1 .

FIG. 1 illustrates an imaging apparatus having an enclosure adapted forsimplifying its installation in a room.

In FIG. 1 , an imaging apparatus assembly (101) has an enclosure (167)that is configured with a particular geometry, and designed to carry animaging apparatus (e.g., 175 illustrated in FIG. 5 ) which is fixed toand specifically aligned with respect to the enclosure (167) such thatwhen the enclosure (167) is attached to surfaces of a room (109) (e.g.,walls (171, 172) and/or ceiling), the imaging apparatus (175) is in apreferred orientation with respect to the room for monitoring.

Preferably, the enclosure (167) of the imaging apparatus assembly (101)is such configured that the imaging apparatus (175) enclosed therein isnot visible to a person (131) within the capturing field of the imagingapparatus (175).

Preferably, the imaging devices as discussed in U.S. patent applicationSer. No. 14/750,403, filed Jun. 25, 2015, published as U.S. Pat. App.Pub. No. 2015/0377711, and entitled “Apparatus and Method forElectromagnetic Radiation Sensing”, U.S. patent application Ser. No.14/788,286, filed Jun. 30, 2015, and entitled “Micromechanical Devicefor Electromagnetic Radiation Sensing”, U.S. patent application Ser. No.14/810,363, filed Jul. 27, 2015, and entitled “Micromechanical Devicefor Electromagnetic Radiation Sensing”, and/or U.S. patent applicationSer. No. 15/188,116, filed Jun. 21, 2016, and entitled “FabricationMethod for Micromechanical Sensors” are used as the imaging apparatus(175) mounted inside the enclosure (167) of the imaging apparatusassembly (101). However, other imaging devices can also be used.

The imaging apparatus (175) disposed within the enclosure (167) may be,for example, a low-resolution thermal imaging apparatus having, forexample, 30×20 thermal infrared pixels to capture the scenery with a lowframe rate (e.g. 1 frame per second, no more than 9 frames per second)and transmitting such imagery wirelessly to a remote receiving unit,while the enclosed, and thus not visible, low-resolution thermal imagingunit and the transmitting unit are powered by a battery unit enclosedwithin the enclosure (167).

For example, the imaging apparatus assembly (101) mounted in a room(109) can be connected to a server (113) and/or a mobile device (117) toform a thermal imaging system illustrated in FIG. 11 and discussedfurther below.

In FIG. 1 , the enclosure (167) has a geometry adapted to simplify theprocess of aligning the orientation of the imaging apparatus assembly(101) with the horizontal direction and the vertical direction of theroom (109), as further illustrated in connection with FIG. 2 . The room(109) has a vertical edge (119) where two walls (171, 172) meet, ahorizontal edge (102 or 104) where a wall (171 or 172) meets the ceilingof the room. The horizontal edges (102 and 104) and the vertical edge(119) meet each other at a ceiling corner (174) of the room (109). Theenclosure (167) has mounting faces adapted to simplify the alignment ofthe orientation of the enclosure (167) with the horizontal and verticaldirections of the room (109).

FIG. 2 shows an imaging apparatus assembly having an enclosure (167)mounted on an edge (119) of two orthogonal walls (171 and 172). Theenclosure (167) has a base face (165) and a top face (164).

FIG. 3 shows a back top-down view of the enclosure (167) illustrated inFIG. 2 , where a vertical back edge (166) of the enclosure is where thevertical mounting faces (162 and 163) meet, and a vertex (161) issuitable for alignment with the ceiling corner (174).

FIG. 5 illustrates an imaging apparatus assembly (101) of FIG. 2 withthe base face (165) of the enclosure (167) being removed (or madetransparent) to reveal the imaging apparatus (175) and its optical axis(177).

In FIG. 2 , the enclosure (167) of the imaging apparatus assembly (101)is designed to enclose and carry an imaging apparatus (175), such as athermal camera for imaging based on infrared radiation, and/or othercomponents of the imaging apparatus assembly (101). The imagingapparatus (175) is fixed to and aligned with respect to thedirections/orientations of the enclosure (167) of the imaging apparatusassembly (101), such that when the directions of the enclosure (167) arealigned with the directions of the room (109), the imaging apparatus(175) has a known orientation with respect to the directions of the room(109).

The enclosure (167) of the imaging apparatus assembly (101) has at leasttwo orthogonal mounting surfaces (162) and (163) illustrated in FIG. 3 .The mounting surfaces (162 and 163) may be perfectly orthogonal to eachother or substantially orthogonal to each other (e.g., having an angleof between 85 to 95 degrees, or 88 to 92 degrees).

It is assumed that the walls (171 and 172) of the room (109) are twovertical planes in the room (109), the floor (e.g., 127 illustrated inFIGS. 10 and 11 ) and the ceiling (e.g., 173 illustrated in FIGS. 8, 9and 10 ) of the room (109) are two horizontal planes in the room (109),the edge (119) where the walls (171) and 172) meet is in the verticaldirection of the room (109) and is perpendicular to the floor (127) andthe ceiling (173) of the room (109), and the edge (102 or 104) where awall (171 or 172) and the ceiling (173) meet is in a horizontaldirection.

Thus, when the imaging apparatus assembly (101) is pushed against anedge (119) where two walls (171 and 172) meet, the mounting surfaces(162 and 163) align with the walls (171 and 172) respectively, whichguides the imaging apparatus assembly (101) into an orientation that isaligned with the directions of the room (109), where the mountingsurfaces (162 and 163) are in parallel with the walls (171 and 172)respectively, the back edge (166) of the imaging apparatus assembly(101) is in parallel with the edge (119) where the walls (171 and 172)meet and in parallel with the vertical direction of the room (109), thetop surface (164) of the imaging apparatus assembly (101) is in parallelwith a horizontal plane of the room (e.g., the floor (127) and/or theceiling (173)), the optical axis (177) of the imaging apparatus (175)has a predetermined direction with respect to the mounting surfaces (162and 163) and the walls (171 and 172), and the optical axis (177) ofimaging apparatus (175) is in a vertical plane in the room and has apredetermined direction with respect to the vertical direction of theroom (109). Similarly, the enclosure (167) can be pushed against theceiling corner (174) for installation at the ceiling corner (174), orpushed against a horizontal edge (102 or 104) for installation at thehorizontal edge (102 or 104) of the room (109).

The top surface (164) can also be optionally configured as a mountingsurface with one or more attachment elements (e.g. adhesive elements).Thus, the imaging apparatus assembly (101) may be pushed against an edge(102 or 104) where a wall (e.g., 171 or 172) and the ceiling (173) meet,or pushed against a corner (174) where two walls (171 and 172) and theceiling (173) meet. The alignment of the orientation of the imagingapparatus assembly (101) with the directions of the room (109) can beeasily achieved via pushing the imaging apparatus assembly (101) againstthe edge (119, 102, or 104) or corner (174) where the imaging apparatusassembly (101) is mounted.

FIGS. 2 and 3 illustrate an example of marking the top surface (164)with an orientation indicator (169), which can be used to avoidinstalling the imaging apparatus assembly (101) sideways where the topsurface (164) is mistakenly pressed against a wall (171 or 172).

FIG. 4 shows an imaging apparatus assembly (101) having an alternativeorientation marker (169) that is on a side face (162) of the enclosure(167) of the imaging apparatus assembly (101). The orientation marker(169) includes an arrow pointing up and the letters “UP” for clarifyingthe intended mounting orientation of the imaging apparatus assembly(101) along the vertical edge (119) of the room (109).

In FIGS. 2 and 3 , the orientation marker (169) contained the letter“TOP” to indicate, as a mounting instruction, that the surface (164) isthe top-surface for mounting the imaging apparatus assembly (101).

In general, the orientation marker (169) may be a graphical indicationof the intended mounting orientation (e.g., arrow) with or withoutletters or numbers as installation instructions. For example, the bottomindication can be marked with feet or shoes, and the top indication canbe marked with a light bulb, sun, cloud, roof, ceiling or any symbolwhich intuitively indicates the correct mounting position.

Preferably, at least one of the mounting surfaces (162, 163, and/or 164)has an attachment element (e.g. adhesive element) to simplify theprocess of installing the imaging apparatus assembly (101). For example,the attachment element may be a double sided adhesive film, whichrequires simply a protective layer to be peeled off by the installer andthe enclosure (167) to be brought into contact with a wall. The adhesivefilm provides a sufficient bond such that no further tools are requiredfor the installation and alignment of enclosure (167) of the imagingapparatus assembly (101). Alternatively, attachment can be achieved vianail, bolt, screw, hole for a wall mounted hook, etc.

The orthogonal mounting faces (162 and 163) illustrated in FIG. 3 allowthe enclosure (167) of the imaging apparatus assembly (101) to bemounted in a vertical edge (119) of two substantially orthogonal walls(171 and 172) of a room (109), as schematically shown in FIG. 2 .

In FIG. 2 , the orthogonal mounting surfaces (162 and 163) are notvisible from the shown perspective as they face walls (171 and 172)respectively. In FIG. 3 , the enclosure (167) of the imaging apparatusassembly (101) is displayed with the orthogonal mounting surfaces (162and 163) facing the viewer of FIG. 3 .

Such a particular geometry of at least two orthogonal mounting faces hasthe advantage that the walls (171 and 172) and/or ceiling (173) of theroom (109) serve to constrain the large number of possible orientationsin which the imaging apparatus assembly (101) could be mounted within aroom (109), when the enclosure (167) of the imaging apparatus assembly(101) is mounted at locations where the walls (171 and 172) and/orceiling (173) of the room (109) join each other at a corner of the room(109). The remaining variables of mounting such an imaging apparatusassembly (101), containing at least two orthogonal mounting surfaces, ina substantially orthogonal vertical edge (119) of a room (109) are thepossible mounting height of the imaging apparatus assembly (101) fromthe floor (127) of the room (109) and the relative orientation (e.g., upvs. down) of the enclosure (167), along the vertical edge (119) of theroom (109).

In any given case the mounting procedure of such an imaging apparatusassembly (101) having the enclosure (167) with the particular geometryis sufficiently simple to be performed by a person without any technicalskills or tools. For example, the installation of the assembly (101) canbe performed by attaching a double sided adhesive tape (e.g., as anadhesive element) to one, or two mounting surfaces (162 and 163) (and/oroptionally surface (164)) of the enclosure (167) of the imagingapparatus assembly (101) and bringing the surfaces into contact with thevertical walls (171 and 172) as illustrated in FIG. 2 , respectively(and optionally in contact with the ceiling (173)). The installation canbe performed as well by bringing the enclosure (167) in contact withonly one mounting face (162 or 163) (whichever contains an adhesive filmfor example) to the wall (171 or 172), respectively, in close proximityof the of the vertical edge (119) of a room. The orthogonal mountingfaces (162 and 163) provide an intuitive shape for an installer to mountit in a substantially orthogonal edge of a room. Further, the solutionallows the enclosure (167) of the imaging apparatus assembly (101) to beinstalled by a person with non-steady or shaky hands or arms. In fact,it requires very low motoric or tactile sensitivity to bring theorthogonal surface into contact with or in close proximity of a verticaledge (119) of a room, thus to mount the enclosure (167) appropriatelywithin a room (109).

Such an enclosure geometry can include for example a tetrahedral shapeas displayed schematically in FIG. 2 and FIG. 3 , where the tetrahedronmay optionally include a third orthogonal mounting surface (164) forcontacting the ceiling (173) during the installation (with or without anadhesive/attachment element). The mounting surfaces (e.g., 162, 163,and/or 164) are planar and either solid or perforated, provided there isenough material to provide an attaching surface to be attached to thewall (171) and/or the wall (172) (and/or the ceiling (173)) of the room(109).

The enclosure (167) of the imaging apparatus assembly (101) may alsoinclude a room-facing base face (165), in the example of a tetrahedralshape opposite to the vertex (161) of the orthogonal faces (referred toas orthogonal vertex (161)). The base face (165) is displayedschematically in FIG. 2 .

Preferably, the base face (165) is not transparent; and the imagingapparatus (175) is not visible to a person (131) within the capturingfield of the thermal camera (175).

FIGS. 6 and 7 show an imaging apparatus assembly having a replaceablebattery unit on its bottom corner.

In FIGS. 6 and 7 , the imaging apparatus assembly (101) is schematicallyshown with a replaceable battery unit (179) on its bottom corner. Anexchangeable or replaceable unit does not need to be on the bottomcorner and can be positioned elsewhere within the enclosure (167). Inthe schematic example of FIGS. 6 and 7 , a replacement battery unit(179) can be detached from enclosure (167) by simply pushing it in once.The attachment mechanism of the replacement battery unit (172) to theenclosure (167) can be through a lock spring, similar to card adapters,where a “push in, push out” mechanism is used to attach or detach a partinto its dedicated socket. The replacement battery unit (179) can beeither replaced by a new battery unit or can have interfaces, such as aUSB interface so it can be recharged through a USB interface.Optionally, a secondary battery unit can be integrated within theenclosure (167) (not visible from the perspective view of FIGS. 6 and 7); thus, if the replaceable battery unit (179) is ejected out of theenclosure (167), power supply for the operation of the imaging apparatus(175) within the enclosure (167) is temporary provided by the secondarybattery while the replaceable battery unit (179) is being replaced.

The base face (165) can have a shape of an equilateral triangle (all 60degree angles—as shown schematically in FIGS. 1-5 , for example) or benon-regular or be a spherical or planar surface.

In other implementations, the base face (165) of the enclosure (167) ofthe imaging apparatus assembly (101) has a curved or spherical shape, asindicated schematically in FIGS. 8 and 9 .

FIGS. 8 and 9 show imaging apparatus assemblies having alternative baseface shapes.

The implementation illustrated in FIG. 8 contains three orthogonalmounting faces (162, 163, and 164), which are not visible from displayedperspective. The mounting faces (162, 163, and 164) and are planar andeither solid or perforated, provided there is enough material to providean attaching surface to be mounted onto wall (171, 172), and/or ceiling(173) of the room (109).

In some implementations, an enclosure (167) having a spherical-shapedbase face (165) can provide up to three orthogonal mounting surfaces andcontain a shape which can be described as an eighth slice of a roundspherical or elliptical geometry (ball, for example) (as illustrated inFIG. 8 ), or a quarter slice of a round spherical or elliptical geometry(as illustrated in FIG. 9 ).

In such an implementation, the enclosure (167) can be mounted in asubstantially orthogonal ceiling corner (174) of the room (109) (e.g.,as illustrated in FIG. 8 ), if the installer is capable of reaching theroom (109) in a convenient way. Otherwise, such implementation can bemounted in the vertical edge (119) of a room (109) with a distance ofseparation between the top surface (164) of the imaging apparatusassembly (101) and the ceiling (173).

In some instances, the ceiling (173) has a strip of material between thetransition of the ceiling to the vertical wall (cove, mold or trayceiling) and in such instances the enclosure (167) can be mounted in thevertical edge (119) of a room (109), as illustrated in FIG. 9 .

In general, the enclosure (167) of the imaging apparatus assembly (101)(e.g. having an overall tetrahedron shape, or aspherical/elliptical/ellipsoidal shape) has two or three orthogonalsurfaces, one or more of which surfaces can be optionally configuredwith adhesive to serve as adhesive mounting surfaces for bonding to awall (171 or 172) and/or a ceiling (173). In some instances, only onemounting surface is configured with an adhesive layer for attaching to awall (171 or 172) or a ceiling (173). By bringing the enclosure (167) ofthe imaging apparatus assembly (101) in contact with a wall in closeproximity to a vertical edge (119) or a ceiling corner (174) of a room(109), the imaging apparatus assembly (101) can adhere to the wallwithout the requirement of any technical tools or skills, in anuncomplicated mounting procedure under a simple set of instructions.

In some implementations the enclosure (167) of the imaging apparatusassembly (101) has only two orthogonal mounting faces (162) and (163)and a curved base face (165) (e.g., having the shape of a portion of aspherical or elliptical surface). The curved based face (165) connectsthe two orthogonal mounting faces (162 and 163) as illustratedschematically in FIG. 9 .

The shape of the enclosure (167) illustrated in FIG. 9 may be describedas a quarter ellipsoid shape, sliced along the long axis of theellipsoidal shape. An orientation indicator can be provided on amounting surface (162 or 163) (not visible in FIG. 9 ) to indicate thedesired mounting orientation of the ellipsoid shape enclosure (167)along the vertical edge (119) of the room (109). For example, theorientation indicator may be a single point with a separate set ofinstruction explaining that when mounting the enclosure (167) in avertical edge of the room that the point shall be closer to the ceilingof the room to ensure proper orientation of the enclosure (167).

In some implementations, the enclosure (167) of the imaging apparatusassembly (101) does not contain more than 3 orthogonal mounting faces(each is orthogonal with respect to remaining mounting faces), to enableand ensure installation simplicity.

One advantage of this mounting procedure is that the vertical wallsserve to constrain the orientation of the enclosure (167), ensuring thatthe orientation of the base face (165) to the room's floor and walls isknown with a high degree of confidence and without the needs to measurethe orientation after the installation.

Further examples of the designs of enclosures can be found in U.S.patent application Ser. No. 29/604,436, filed May 17, 2017 and entitled“Camera Enclosure”, the disclosure of which is hereby incorporatedherein by reference.

FIG. 10 illustrate geometrical relations among the mounting positing ofan enclosure (167), the orientation of the optical axis of an imagingapparatus housed within an enclosure (167), the field of view of theimaging apparatus, and the space within the room that can be captured inimages obtained from the imaging apparatus housed within the enclosure(167).

Within the thermal image assembly (101), the imaging apparatus (175) ismounted to have a predetermined orientation with respect to itsenclosure (167) (e.g. a desired alignment of its optical axis (177) withrespect to the base face (165)), such that when the enclosure (167) ofthe imaging apparatus (175) is mounted in alignment with the walls (171,172) and/or ceiling (173) of the room (109), the imaging apparatus (175)achieves substantial alignment with the area of interest in the room(109). This mounting of the imaging apparatus (175) with respect to theenclosure (167), in conjunction with the alignment of the base face(165) constrained by positioning the enclosure (167) of the thermalimage assembly (101) in a room corner (174) or room's vertical edge(119), ensures the imaging apparatus (175) views the room (109) on awell-defined axis (177) with respect to walls (171 and 172) and thefloor (127) of the room (109).

The desired orientation of the axis (177) of an imaging apparatus (175)with respect to the enclosure (167) depends on a number of factors, forexample to best serve the imaging apparatus and application, to achievea desired apparatus coverage or to target a particular room geometry.

In one implementation, the mounting of the imaging apparatus (175)within the enclosure (167) is arranged so that the imaging axis (177)equally bisects the angle between the two mounting walls (171 and 172)to the horizontal.

In one implementation, the mounting of the imaging apparatus (175)within the enclosure (167) is arranged so that the imaging axis (177)equally is perpendicular or is tilted with respect to the vertical edgeof the room.

In some instances, the imaging apparatus (175) has a field of view(capturing viewing angle) of 90 degrees or more. When such an imagingapparatus is used, a symmetrical orientation of the image apparatus(e.g., the imaging apparatus (175)) fixation within the enclosure (167)in an orthogonal room can result in substantially full room coverage orcoverage of a reasonable proportion of the room.

In some instances, the enclosure (167) houses two or more imagingapparatuses (e.g., imaging apparatus (175)). In such instances theenclosure (167) includes a fixation mounting for each imaging apparatus(175) which allows the optical axis (177) of each to be fixed relativeto the base face (165). In one possible implementation, the optical axes(177) of the imaging apparatuses can be distributed evenly in ahorizontal plane and/or a vertical plane.

In some instances, the optical axis (177) may have an inclination anglefrom the horizontal plane that is parallel to the ceiling plane (173) orthe floor plane (127). The mounting of the enclosure (167) can beperformed at a height at or above a typical human's head or even in theceiling corner (174) of a room (109), so that the imaging apparatus(175) has an optical axis (177) being oriented towards the room,containing an inclination angle relative to the horizontal plane, withthe apparatus “looking down” on the room (109). The orientation marker(169) on the enclosure (167) functions as an indicator for ensuring thatthe enclosure (167) is in the correct orientation for the imagingapparatus to be facing towards the room and towards the floor (127) ofthe room (109).

In general, multiple imaging apparatuses can be housed within theenclosure (167) of the imaging apparatus assembly (101), depending onthe size of the field of view of the imaging apparatuses. For example,when an imaging apparatus has a field of view of 90 degrees or more isused for corner or edge mount, one imaging apparatus may be sufficient.When imaging apparatuses each having a limited field of view (e.g. 30degrees), an array of imaging apparatuses (e.g. 3×3) can be configuredto stitch together the fields of views to cover the room.

The problem of the imaging apparatus (175) (or multiple thereof) withinthe enclosure (167) being visible to a person standing in front of theenclosure (167) is solved by a base face (165) which is visually opaqueor translucent from the outside of the enclosure (167).

Such a visibly opaque surface could be an infrared-transparent materialif the imaging apparatus (175) inside the enclosure (167) detects oremits in the infrared band (e.g., as in the related applicationsidentified above). In some implementations such a visibly opaque, butinfrared-transparent surface can be made out of polymer material, suchas polyethylene (PE) or polypropylene (PP). Such polymer materialsappear white and non-transparent in the visual band for the human eye,but can be transparent in the infrared band. Other visuallynon-transparent, but infrared transparent materials include Germanium(Ge) or Silicon (Si). These materials appear in the visual band, for thehuman eye, “black” and visible light cannot pass through such materialsdue to no transmission in the visual band. In another instance it may bea partially transparent mirror (one-way or two-way mirror; a visuallytransparent material coated with a thin metallic layer) where a personfacing the plane sees a reflective surface as the base face (165), whilethe imaging apparatus (175) can image through the partly visuallytransparent surface.

In at least some embodiments, the enclosure (167) of the imagingapparatus (175) is configured for simplicity of mounting procedure, withthe fixed, “self-aligned” viewing angle of an imaging apparatus (175)configured within the enclosure (167) that has an orientationconstrained by its mounting surfaces contacting the walls of asubstantially orthogonal vertical edge of a room. Thus, if theparticular orientation of the imaging apparatus (175) within theenclosure (167) is known, the angle of the field of view (185) of theimaging apparatus (175) is known, and the approximate mounting height(123) is known, the space that is monitored by the imaging apparatus canbe computed to determine whether it includes the one or multiplestanding subjects (131) having a height (183) and positioned with adistance (181) and an angle within the horizontal plane of the room(109) (e.g., relative to the walls (171 and 172).

On the other side, in order to provide the desired space covered by thethermal imaging assembly (101), the desired mounting height (123) can becomputed from the distance (181) between the furthest subject havingheight (183), the orientation of the field of optical axis (177)relative to the imaging apparatus assembly (101), and the angle of thefield of view (185) of the imaging apparatus (175). The orientation ofthe field of optical axis (177) relative to the enclosure (167), and theangle of the field of view (185) of the imaging apparatus (175) ispredefined by manufacturing of enclosure (167) in one embodiment.

In one implementation, the set of mounting instructions for auser/installer instructs the user to peel off protective layer from adouble sided adhesive tape, which is already pre-installed by default onmounting surface (162 and/or 163) of the enclosure (167) of the imagingapparatus assembly (101), and mount the enclosure (167) into asubstantially orthogonal vertical edge (119) of a room (109), at aheight (123) of approximately 6 feet or higher above the floor (127).

In some implementations, the imaging apparatus (175) inside theenclosure (167) has, for example, about 30×20 pixels with a horizontaland vertical field of view (185) of slightly larger than 90 degrees. Theimaging apparatus (175) being battery operated, activated by the userby, for example, pushing a button on the mounting surface (162), orreleasing a contact-stopping tape from battery compartment, oractivating remotely via a handheld computer (e.g., 117)). The imagingapparatus assembly (101) streams the recorded footage wirelessly to areceiver (e.g., using a wireless transmitter for wireless local areanetwork, wireless personal area network, Wi-Fi, Bluetooth, Zigbee, radiotransmission, cellular communications, etc.). The low resolution of theimaging apparatus (175) provides privacy protection to occupants of theroom. In such an implementation, the base plane (165) can be a white,visually non-transparent film, made out of a thin PE-membrane, hidingthe content of the enclosure (167) and in particular the imagingapparatus (175).

The orientation of the imaging apparatus (175) inside the enclosure(167) can be such that it is symmetric in the horizontal plane andsymmetric to the vertical plane, where the horizontal plane can bedefined as substantially plane parallel to the floor (127) of the room(109) and the vertical plane can be defined as substantially planeparallel to one of the mounting walls of the room. For example, theorientation of the imaging apparatus (175) inside the enclosure (167)can be such that its optical axis (177) is 45 degrees downward relativeto the back edge (166) that joins the faces (162 and 163) and have equalangles relative to the faces (162 and 163). For example, the orientationof the imaging apparatus (175) inside the enclosure (167) can be suchthat its optical axis (177) is aligned in the plane that bisect theenclosure (167) vertically (e.g., passing through the vertical edge thatjoins the faces 2 and 3) and have a predetermined angle (e.g., 45degrees) relative to the vertical edge. With the known orientation ofthe camera preset at manufacture, its preset field of view (185) and itsapproximate mounting height (123), the captured image can be analyzedfor the position of a subject (131) within the field of view (185) andthe height (183) of the subject (131) as well as the width, asschematically shown in a cross-section 2-dimensional view of FIG. 10 .

In FIG. 10 , when the mounting height (123) of the imaging apparatusassembly (101) is known, for example by any of the methods discussedabove in connection with FIGS. 11-13 , and the optical axis (177) andthe field of view (185) are known from the design and manufacture of theenclosure (167) of the imaging apparatus assembly (101), then theobservable spatial position and the distance (181) between a subject orobject (131) within the field of view of the apparatus 20 can bedetermined.

In FIG. 10 , the dotted lines from the imaging apparatus assembly (101)reversely project the pixels to the floor (127) and the wall on theopposite side. The thermal radiation between adjacent dotted lines ismeasured by a corresponding pixel in an imaging apparatus (175) in theimaging apparatus assembly (101). Thus, the spaces between the dottedlines represent the spaces imaged by the corresponding pixels.

For example, the thermal radiation projected to the imaging apparatusassembly (101) between the dotted lines (188 and 189) is measured bypixel 1; and the thermal radiation projected to the imaging apparatusassembly (101) between the dotted lines (187 and 188) is measured bypixel 2; etc. The thermal intensity measured by the pixels 1, 2 andothers form a vertical line (186) of pixels in a thermal image. Thethermal image (131) of the subject or object (131) is represented by theshaded pixels (183). For the given mounting height (123) and the fieldof view (185), a count of pixels (181) up to the bottom of the thermalimage (133) of the object (131) corresponding to a determined horizontaldistance (181) between the location of the subject or object (131) andthe edge (119) on which the imaging apparatus assembly (101) is mounted.The count of the shaded pixels represents the height (183) of thethermal image (133) of the subject or object (131) in the imagecoordinate system (139), which corresponds to the real world height ofthe subject or object (131) above the floor (127) of the room (109) inview of the mounting height (123). The geometrical relation can also beused in reverse direction to determine the mounting height (123) basedon the real world height of the subject or object (131) and the count ofthe shaded pixels that represents the height (183) of the thermal image(133) of the subject or object (131) at a location identified by thecount of pixels (181) below the shaded pixels.

In FIG. 10 , the one-dimensional vertical pixel row (186) shows how thesubject (131) in the room (109) appears in the thermal image captured bythe imaging apparatus assembly (101). The radiation from the subject(131) causes the shaded pixels to be measured to have a temperatesignificantly different from the other areas that are measured by thenon-shaded pixels. The non-shaded pixels represented the portion of theroom measured at the room temperature; and the shaded pixels representedthe elevated surface temperate of the subject (131) over the roomtemperature.

In FIG. 10 , the vertical row of pixels (186) are identified as with“Pixel 1”, “Pixel 2”, etc., which correspond to the imaged spaces markedcorresponding with “Pixel 1”, “Pixel 2”, etc.

Assuming the subject (131) is standing vertically within the room (109),his or her height (183) and position (181) can be determined bytrigonometric relations. Analogue example is valid for the horizontaldimension, which allows the determination of the subject's or object'sposition within the horizontal dimension and its width. This is validfor any object having a temperature different from the room temperaturein case of imaging in thermal infrared.

For example, hot-spots or cold-spots can be allocated by knowing theirposition and their relative size, in addition to its relativetemperature. Hot-spots could include hazardous items such as for examplean iron that was accidently forgotten to be turned off by a user andleft was unattended and can be a potential fire or safety hazard, orcold-spots could include an open window when very cold air is streaminginto the room that was forgotten to be closed by a person. Manycold-spots and hot-spots can be detected by a low resolution thermalimaging apparatus. Accordingly, 3 dimensional information of the viewingscenery can be reconstructed of the recorded image of the imagingapparatus assembly (101).

The example of FIG. 10 is simplified to a cross sectional,two-dimensional case with one vertical pixel row (186) of 20 pixels,representing the vertical imaging capacity of the imaging apparatusassembly (101) in such an example. The imaging apparatus has preferablya viewing capacity of an array of rows, equivalent to a matrix of pixelsof, for example, 30 pixels in the horizontal direction by 20 pixels inthe vertical direction.

Optionally, additional functions may be integrated within the enclosure(167) of the imaging apparatus assembly (101), such as a decorativesurface on the visible side of the base face (165), lighting, Wi-Fiaccess point/repeater, etc.

Optionally, the enclosure (167) of the imaging apparatus assembly (101)can have rounded corners/edges and a rounded vertex (161) for easierfit/mount into a rounded vertical edge or a rounded corner of a room.

Optionally, an adapter enclosure is permanently fixated and mounted onthe walls and the enclosure (167) of the imaging apparatus assembly(101) containing the imaging apparatus (175) and/or other parts thereof(e.g. battery) is attached to the adapter enclosure such that theimaging apparatus assembly (101) can be easily replaced without the needof demounting the entire assembly.

Optionally, any part of the imaging apparatus assembly (101) disposedwithin its enclosure (167), such as one or the multiplicity of theimaging apparatus (175), the battery, the wireless module, theelectronic board, etc. can be designed to be exchangeable or replaceablewithin the enclosure (167), while the enclosure (167) can be permanentlyfixated and mounted on the walls without the need of demounting theentire assembly. For example, a battery module can be replaceable in away as illustrated in FIGS. 6 and 7 ; and other replaceable modules canbe similarly configured for the imaging apparatus (175), an optionalwireless module, etc.

In some instances, wedges are provided for mounting between theenclosure (167) of the imaging apparatus assembly (101) and the wall(s)(e.g., 171 and 172) and/or the ceiling (173), if walls and/or theceiling of the room (109) are not substantially orthogonal to eachother.

At least some embodiments disclosed herein provide a user friendly wayto determine the installation configuration of a thermal imagingassembly, based on the thermal images captured at the time of thecalibration of the thermal imaging assembly in a thermal imaging systemand user inputs provided in connection with the thermal images. The userinputs train the thermal imaging system to gain knowledge about theenvironment in which the thermal imaging assembly is installed andconfigured to monitor. The configuration parameters and the knowledgeabout the environment are used subsequently to interpret the imagesobtained at a time of monitoring service and generate monitoringoutputs, such as identifying the presence, location, and/or activitiesof humans, telling adults, children, and pets apart, etc.

For example, the user may provide the height of a person (e.g., theuser) detectable in the thermal images during theinstallation/calibration of the thermal imaging system to allow thesystem to compute a mounting height of the thermal imaging assembly.Other user inputs may include an indication of the time instance whenthe user is at a point of interest (POI) (e.g., room corner, door),identification of a POI, etc., to allow the system to learn thelocations of the POI in the imaging coordinate system, where the POI maynot be visible or recognizable from the thermal image directly.

During the installation/calibration, the system may instruct the user toperform activities, such as walking away or to the camera, going to apoint of interest, walking along a path way in an area monitored by thecamera, walking in an area heavy for foot traffic, etc. The useractivities generate thermal images from which the system learns thegeographical configuration of the monitored environment.

Based on the user inputs and/or the thermal images collected during theinstallation/calibration, the system computes configuration parameters,such as the mounting height of the thermal imaging assembly, a ratio ormapping between a size in the image and a size of a person/object in themonitored area, and the identification of POIs in images captured by thethermal camera. The system bookmarks the locations, paths, and/or areasof interest as knowledge about the environment in which the thermalimaging assembly is installed and configured to monitor.

For example, a mobile application is configured in one embodiment to askthe user to enter the height of the user captured in a thermal imagepresented on the mobile application. Once the mobile application detectsthe user in the image, the application may instruct the user to performan act, such as entering the height of the user, or going to a point ofinterest, such as a corner of the room, a door or window of the room,etc. The mobile application (or a remote server) extracts locationand/or size data from the thermal images of the user performing the actand correlate the instruction and/or optional input from the user todetermine configuration parameters, such as the mounting height of thethermal camera, the location of the point of interest in the thermalimage coordinate system, a location mapping between the thermal imagecoordinate system and a coordinate system aligned with the room, a sizemapping between the object sizes measured in the thermal imagecoordinate system and the real world object sizes in the room coordinatesystem.

FIG. 11 shows a thermal imaging system according to one embodiment.

In FIG. 11 , the thermal imaging system includes an imaging apparatusassembly (101) and a server (113) that processes the thermal imagescaptured by the thermal camera included in the imaging apparatusassembly (101) and provides services based on the thermal images.

In FIG. 11 , the imaging apparatus assembly (101) communicates thethermal images to the server (113) via a wireless access point (111) anda computer network (115) (e.g., a local area network and/or theInternet). A mobile device (117), such as a smartphone, a tabletcomputer, a laptop computer, or a personal media player, has a mobileapplication installed therein to communicate with the imaging apparatusassembly (101) and/or the server (113) for the calibration, setup,and/or the usage of the thermal imaging system.

In some instances, the imaging apparatus assembly (101) communicates theraw footage (e.g., via a wireless connection or a wired connection) tothe mobile device (117) and/or the server (113) without performing anyimage processing within the enclosure (167) of the imaging apparatusassembly (101). A host device (e.g., the mobile device (117) or anothercomputer in the room (109), or in the vicinity of the room (109)), orthe server (113) that is remote to the installation site, performs imageprocessing to provide the user interfaces and/or compute configurationparameters, as discussed below in detail. In some instances, the server(113) is implemented using the cloud computing third-party serviceprovided via serverless architectures.

In FIG. 11 , the imaging apparatus assembly (101) is mounted at alocation in an environment, such as a room (109), that is beingmonitored by the imaging apparatus assembly (101).

Preferably, the imaging apparatus assembly (101) is mounted on avertical edge (119) where two walls (171 and 172) of the room (109) meeteach other, a horizontal edge (102 or 104) where a wall (e.g., 171 or172) and the ceiling of the room (109) meet each other, or a corner(174) of the room (109) where two walls (171 and 172) of the room (109)meet the ceiling of the room (109). Alternatively, the imaging apparatusassembly (101) may be mounted at other locations, such as on a locationon a surface of a wall (e.g., 171 or 172) or ceiling, or any arbitraryplace within the scenery. For example, the imaging apparatus assembly(101) can be configured to be mounted on the ceiling of a room fortop-down monitoring; and the imaging apparatus assembly (101) may bemounted on and/or with holders and/or devices, such as IP camera,passive infrared sensor (PIR), etc.

Preferably, the imaging apparatus assembly (101) has an enclosure (167)or housing that has surfaces adapted to be aligned with the surfaces ofwalls (e.g., 171 or 172) and/or the ceiling of the room (109). Thus, thealignment of the orientation of the imaging apparatus assembly (101)with respect to the vertical direction and the horizontal direction canbe easily achieved by pressing two or more mounting surfaces of theenclosure (167) or housing of the imaging apparatus assembly (101)against the flat surfaces of the wall(s) (171, 172) and/or the ceilingof the room (109).

For example, the external mounting surfaces of the enclosure (167) orhousing of the imaging apparatus assembly (101) of one embodiment havepre-applied adhesive materials covered with protection strips that canbe peeled off to reveal the adhesive materials for mounting. When theenclosure (167) or housing of the imaging apparatus assembly (101) ispressed against the edge (119) or corner (174) at the mounting location,the mounting surfaces of the enclosure (167) or housing of the imagingapparatus assembly (101) align with, and adhere to, the surfaces of thewall(s) and/or ceiling. The alignment of the mounting surfaces of theenclosure (167) or housing with the wall and/or ceiling surfaces resultsin the alignment of the imaging apparatus assembly (101) with respect tothe horizontal and/or vertical directions in the room (109).

In some instances, the enclosure (167) or housing of the imagingapparatus assembly (101) is fixedly attached to the mounting locationvia elements, such as nails, screws, etc.

When the imaging apparatus assembly (101) is mounted in the room (109)with proper horizontal and vertical alignment, the camera in theassembly (101) has a known orientation with respect to the orientationof the room (109). However, the mounting height (123) (e.g., thevertical distance from the floor (127) to the imaging apparatus assembly(101)) is not yet known to the imaging system.

The mounting height (123) may be measured (e.g., using a measuring tape)and provided to the system via a user interface, such as a graphicaluser interface provided by a mobile application running on the mobiledevice (117). In some instances, the orientation of the imagingapparatus assembly (101) can be determined automatically from tiltsensors and/or other sensors (e.g., a set of accelerometers and/or a setof magnetic sensors).

Alternatively, when the imaging apparatus assembly (101) has two camerasmounted within their enclosure (167) (or adjacent room corners) with aknown distance to each other, the server (113) can use a stereoscopicvision provided by the cameras to determine the mounting height astereoscopic view of one or more reference object(s).

Alternatively, when the imaging apparatus assembly (101) has a distancemeasurement sensor that measures a distance based on the time of flight(TOF) of a signal, the imaging apparatus assembly (101) can measure itsmounting height from the floor plane (127) automatically. The TOF can bemeasured based on ultrasonic signs, or radio frequency signals.Alternatively, the mounting height may be measured via barometric and/ormotion sensors. In some instances, the imaging apparatus assembly (101)includes sensors and/or devices, such as GPS receivers to the determinedthe location of the imaging apparatus assembly (101), magnetic sensorsfor determine the orientation of the imaging apparatus assembly (101)relative to the magnetic field of the Earth, light and/or audio devicesfor provide visual and/or audio feedback and/or alerts, air qualitymonitoring devices, etc.

In a preferred embodiment, the imaging system determines the mountingheight based on measuring the size of a reference object (131) in athermal image and receiving an input of the real world size of thereference object.

For example, the reference object (131) in FIG. 11 has a top point (106)and a bottom point (105) that are captured in the thermal imagegenerated by the imaging apparatus assembly (101). The thermal image ofthe monitored area illustrated in the projected image plane (103) has animage (133) of the reference object (131) with a corresponding top point(108) and a corresponding bottom point (107). A measurement of the sizeof the image (133) of the reference object (131) and an inputidentifying the real world size of the reference object (131) can beused to compute the mounting height (123), as further discussed below inconnection with FIG. 12 .

For example, the reference object (131) can be the person installing,calibrating, and/or setting up the imaging apparatus assembly (101), oranother person in the monitored area of the room (109); and the heightof the person is the real world size of the reference object (131) inthe computation of the mounting height (123). Such an approach greatlysimplifies the process to calibrate and/or set up the thermal imagingsystem.

FIG. 12 illustrates a method to measure mounting configurationparameters of a thermal imaging camera according to one embodiment. Forexample, the method of FIG. 12 can be used to determine the mountingheight of the imaging apparatus assembly (101) of the image systemillustrated in FIG. 11 .

In FIG. 12 , the camera at the mounting location (121) has a determinedmounting angle (125) with respect to its housing or enclosure (167) thatis aligned with the room coordinate systems. The measurement of sizes inthe captured thermal image is performed in a predetermined projectedimage plane (103), or image coordinate system, that corresponds to apredetermined mounting height (122) that has a fixed geometricalrelation with respect to the image plane (103), as defined by themounting angle (125). The predetermined mounting height (122) can beconsidered a reference mounting height from the reference floor (126);and a reference height (130) in the vertical direction is projected tohave an image (133) in the projected image plane (103).

In FIG. 12 , when the real world reference object (131) has the samesize as the image (133) in the imaging plane (103), the ratio betweenthe reference height (130) and the height of the real world referenceobject (131) is the same as ratio between the reference mounting height(122) and the real world mounting height (123) to the floor (127) onwhich the object (131) stands. Thus, the reference mounting height (122)can be scaled up to obtain the real world mounting height (123) based onthe ratio between the reference height (130) and the height of the realworld reference object (131). The reference height (130) can bedetermined from the size and location of the image (133) and themounting angle (125) of the camera.

In one implementation, from the mounting angle (125), the referencemounting height (122), and the size and location of the image (133), aformula is derived to compute the reference height (130). The ratiobetween the height of the reference height (130) and the height of thereal world reference object (131) can be used to scale the referencemounting height (122) to the real world mounting height (123).

In another implementation, the imaging apparatus assembly (101) ismounted in a reference room at a reference height (122). References ofobjects (e.g., 130) of different heights are positioned at the locationof the object (130) illustrated in FIG. 12 to obtain images (e.g., 133)of different sizes and thus establish a mapping between the referenceheights and the image sizes. When the real world object (131) generatesan image size at the location in the image plane (103), the mapping canbe used to look up the reference height (130) in the reference room. Thereference mounting height (122) can then be scaled to the real worldmounting height (123) to the floor (127) according to the ratio betweenthe reference height (130) looked up from the mapping according to thesize of the image (133) and the height of the real world referenceobject (131) that generates the image (133) of the same size in theimage plane (103).

In one embodiment, the camera in the imaging apparatus assembly (101)has a limited field of view. For example, in FIG. 12 , the camera maynot be able to capture the area that is closer to the edge (119) thanthe line (129). Thus, when the object (131) generates image (133) thatborders on the boundary of the image frame, the system may not be abledetermine whether the image captures the entire object (131). Thus, themobile application running in the mobile device (117) may provideinstructions to move the object (131) such that the image (133) leavesthe boundary of the image frame and the image (133) does not have pixelsat the boundary of the image frame. As soon as the image (133) leavesthe boundary, the system captures an image (133) of the object (131) anddetermines the reference height (130) that generates the same image sizeat the location.

In general, the mobile application may provide a display of the thermalimage captured by the thermal camera assembly and provide instructionsto guide the movement of the object (131) to a specific location in theroom such that the object (131) is shown to be standing in a particularlocation in the thermal image. The size of the object (131) in the imageis then used in combination with the real world size of the object (131)to compute the mounting height (123).

FIG. 13 shows a user interface to obtain a user input to determine amounting height of a camera according to one embodiment. For example,the user interface of FIG. 13 can be implemented on the mobile device(117) in the thermal imaging system of FIG. 11 to compute the mountingheight using the method of FIG. 12 .

In FIG. 13 , after the imaging apparatus assembly (101) is mounted atthe mounting location (121), the camera is configured to establishcommunications with the server (113) and/or the mobile device (117) toprovide a thermal image of the monitored area of the room (109) fordisplay on the mobile device (117). The thermal image (141) presented onthe mobile device (117) (e.g., using a mobile application running in themobile device (117)) has a thermal image (133) of an object (131) in theroom (109) that has a temperature significantly different from the roomtemperate. For example, the object (131) is a person in the room (109),such as the installer or owner of the imaging apparatus assembly (101),or another person in the monitored area of the room (109). The userinterface requests the user of the mobile device (117) to enter the realworld height of the object (131) (e.g., a person) identified in thethermal image (141).

In some instances, when the thermal image (141) captures the thermalimages of multiple objects, the user interface allows the user of themobile device (117) to select an object (131) and specify the height ofthe selected object (131).

The thermal imaging system is configured to measure (e.g., by the mobileapplication running in the mobile device (141), the server (113), or theimaging apparatus assembly (101)) the height (137) of the thermal image(133) in an image coordinate system (139).

In some instances, the thermal imaging system is configured to measurethe height (137) when the thermal image (133) is at a specific locationin the image coordinate system (139) (e.g., when the thermal image (133)of the object (131) stands at a specific location in the imagecoordinate system (139)). In such implementations, the user interfacemay provide instructions to guide the movement of the object (131) suchthat the thermal image (133) stands at the specific location marked inthe image (141) presented on the mobile device (117).

In other instances, the thermal imaging system can measure the height(137) to compute the mounting height (123) using the height (143)provided by the user of the mobile device (117), without requiring theobject (131) to go to a specific location, as long as the thermal image(133) of the object is captured in the image (141) in entire (e.g., noportion of the object (131) is located in a blind spot of the imagingapparatus assembly (101)).

In some instances, the mobile device (117) instructs the user to movethe object (131) (e.g., the user) around in the room, such that theheight (137) of the thermal image (133) of the object (131) can bemeasured in a plurality of different locations to compute the mountingheight (123) from the measurement of the corresponding locations. Thecomputed results of the mounting height (123) can be combined (e.g.,from an average of the results, or a weighted average the results) forimproved accuracy.

After the mounting height (123) is determined, the thermal imagingsystem can map the coordinates in the image coordinate system (139) (ofpoints with known or inferred height) to the coordinates in the room(109).

FIGS. 11-13 discussed the measuring of the mounting height of a thermalcamera (imaging based on IR radiation). The technique can also besimilarly extend to the determination of the mounting height of a camerathat images based on lights that are visible to human eyes.

While an image of the room (109) captured based on visible lights mayshow features of the room (109) that can be used to automaticallydetermine the layout of the room (109) (e.g., the floor boundaries andthe location of points of interest, such as the door, window,furniture), a thermal image of the room (109) typically does not havesufficient features that can be used to identify the layout of the room(109), even to a human, especially when the thermal image has a lowresolution to protect the privacy of the occupants of the room (109).

In some instances, the imaging system as illustrated in FIG. 11 isconfigured to automatically calibrate and/or re-calibrate the mountingheight (123) and/or other configuration parameters (e.g., POI locations)based on a statistical analysis of the thermal images of humans observedover a period of time.

For example, after the imaging system detects the thermal images of anumber of people that have been to the monitored area (e.g., room (109))over a period of time, the system computes a statistical distribution ofthe relative height of the people detected in the period of time. Themounting height can be scaled to match the distribution of the height ofthe detected people with a known distribution of height of people (e.g.,people in the same geographical region and/or having ages within anexpected range for people visiting the monitored area). A mountingheight computed from the height distribution can be used in place of amounting height computed from an input of the height of a user detectedduring the installation process, or to cross check and/or improve themounting height computed during the installation process. Thus, thecalibration accuracy can improve over time based on the monitoringresults of the imaging system.

In some instances, an object of a known height in the monitored area(e.g., room (109)) can be detected in certain time periods of theservice of the imaging system. For example, when there is a significanttemperature difference between the room (109) and the environmentoutside the room (109), opening of the door at some time instances wouldallow the imaging apparatus assembly (101) to generate an image wherethe door opening area has a thermal image recognizable in the imagecoordinate system (e.g., the opening area has a temperature of theenvironment outside the room (109) while the wall on which the door ismounted has the temperature of the room). The height of the door canthus be determined from the thermal image of the door opening area tocalibrate the mounting height of the imaging apparatus assembly (101)against a known or standard door height.

For example, after the imaging apparatus assembly (101) having anenclosure (167) illustrated in FIGS. 1-7 is placed in a monitored area(e.g. room (109)) to capture the scenery, a calibration/training processis configured to allow the thermal imaging system (e.g., as illustratedin FIG. 11 ) to understand the true geometric data from the capturedfootage, and recognize points of interests (POI) which may not bedetectable in the imaging apparatus capturing band. For example, whenthermal imaging in the band of infrared radiation looks at the sceneryof uniform room temperature, the POIs are not detectable due to the lackof distinct temperature contrast or temperature differences.

When an enclosure (167) illustrated in FIGS. 1-7 is used, a set ofconfiguration parameters are known (from factory configurations) withoutthe need for measurements from the installation site. Such configurationparameters may include: image size and aspect ratio, imaging lensparameter (e.g., the Field of View), the mounting angles in a3-dimensional space relative to some reference point/plane/coordinatesystem of the scenery (e.g. room (109)). The mounting height (123) canbe determined using the method illustrated in FIGS. 11-13 .

The mounting height can be determined using other technical solutionsthat use additional resources. For example two cameras can be mountedwithin the enclosure (167) with a known distance to each other; andthrough stereoscopic vision the referencing of objects can bedetermined. For example, a distance measurement sensor, such as time offlight (TOF) sensor, can be included in the imaging apparatus assembly(101) to measure its mounting height from the floor plane (127)automatically. For example, the installer may be instructed to measurethe mounting height (123) using a measuring tape and enter themeasurement via a user interface provided by the mobile device (117)configured for the calibration of the thermal imaging system.

Once the configuration parameters are known, the thermal imaging systemhas a mapping from the image coordinate system (139) and the roomcoordinate system for a true geometric understanding from the capturedfootage.

For example, the mobile device (117) is configured to establish acommunication connection (e.g., via an access point (111) for a wirelesslocal area network) with the imaging apparatus assembly (101) and/or theserver (113) for the calibration of the imaging apparatus assembly (101)installed in the room (109). The mobile device (117) provides a userinterface to instruct the user to move around within the room such thatthe user is fully visible in the thermal image captured by the imagingapparatus assembly (101) and instruct the user to be on the floor (127)in a standing position. Optionally, the user interface may instruct tothe user to go to one or more preferred locations where the user isfully visible in the thermal image. The mobile device (117) receives aheight of the user (e.g., via a user interface, or from a data sourcethat has the height of the user).

In general, from the full thermal image (133) of the user (131) standingvertically on the floor plane (127), the system measures the size of thethermal image (133) and computes the mounting height (123) of theimaging apparatus assembly (101) to match the real world height of theuser (131) as projected to the location and size of the thermal image(133) of the user (131) in the image coordinate system (139). It is notnecessary to know the exact position and/or distance of the user (131)relative to the imaging apparatus assembly (101) or the edge (119) onwhich the imaging apparatus assembly (101) in order to compute themounting height (123).

In some instances, the user or object (131) having a known height may beobstructed by other objects which could falsely show a different heightof the user or object (131) in the image than the actual height in thethermal image (e.g., in particular in low resolution thermal infraredimage). For example, a hot object or other subject of same temperatureappears to be above the user or object (131) in the image produced bythe imaging apparatus assembly (101), which makes the user or object(131) subject appear taller in the thermal image. By instructing theuser (131) to walk around in the monitored area, the system can detectthe thermal image of the user (131) in relation with the thermal imagesof other objects having temperatures that are significantly differentfrom the room temperature and thus identify a preferred location to takea measure of the size of the thermal image (133) of the user (131) forthe computation of the mounting height (123). In the preferred location,the thermal images of other objects do not interfere with the measure ofthe size of the thermal image (133) of the user (131) in the imagecoordinate system (139). Optionally, the user interface of the mobileapplication running in the mobile device (117) shows the thermal imagefrom the imaging apparatus assembly (101) in real time, such that theuser may verify that the user is standing in a location in a room wherethe thermal images of other objects do not overlap with the thermalimage (133) of the user (131) in a way that affects the accuratemeasurement of the height of the thermal image (133) of the user (131)in the image coordinate system (139).

Calibration through the thermal image of a human having a known heightis one example. In general, any object of known height/geometry, whichcan be detected by the imaging apparatus of the imaging apparatusassembly (101) can be used to as the reference object (131) in theimaging scenery, such as a cup of hot water, a bottle of iced water.Typically, the human body forms an ideal reference object in thermalinfrared due to a good contest from a typical room temperaturebackground and a size similar to the objects to be monitored in thescenery. Thus, the use of a human body as a reference offers simplicityand easiness in installation and calibration, which in fact does notrequire any know-how of technical expertise or any otherobjects/apparatuses/aids to perform this type of referencing.

Preferably, the communication connectivity between the imaging apparatusassembly (101) and a centralized remote server (113) allows the storageof the calibration information of the imaging apparatus assembly (101)installed in a monitored area (e.g., room (109)) in the cloud tofacilitate cloud computing. Such an arrangement enables a very smoothuser experience and friendly interface (e.g., implemented via a mobileapplication running in a mobile device (117)). Some of the computationand/or resource intensive tasks can be performed on the mobile device(117) and/or on the server (113). Thus, the cost of the imagingapparatus assembly (101) can be reduced.

In a typical installation process, the mobile device (117) (or aninstruction manual) instructs the user to:

1. activate the imaging apparatus assembly (101) (e.g. by peeling ofprotection strip between battery contact and battery, or pushing abutton to switch device on, supply power through cable, etc.);

2. establish a communication connection to the imaging apparatusassembly (101) (e.g., by using a mobile application installed on themobile device (117), such as a smartphone, a tablet computer, or apersonal media player, to scan a code which is placed on or associatedwith the imaging apparatus assembly (101) that has a unique device IDused for establishing an authorized communication connection, such as awireless personal area network connection (e.g., Bluetooth connection orNear Field Communication connection), and by using the mobileapplication and the connection to the imaging apparatus assembly (101)to configure the connection to the access point (111) and the server(113) over the network (115) and configured the imaging apparatusassembly (101) in a user account);

3. optionally, enable a connection to a cloud-based computing & storagesystem (e.g., a connection between the imaging apparatus assembly (101)and the server (113), via internet, either standalone from imagingapparatus assembly (101) or through a low power communication connectionwith a hub, using Bluetooth, Zig-Bee, etc. in order to preserve energy,where the hub is connected to the internet);

4. optionally, configure the mobile device (117) to perform at leastsome of the functions of the server (113) in storing and/or processingthe calibration information;

5. mount the imaging apparatus assembly (101) in an edge (119) or acorner (174) of a room (109), preferably at a location above a headheight for a desirable coverage of the monitored area, where theenclosure (167) of the imaging apparatus assembly (101) as illustratedin FIGS. 1-9 ) simplifies the alignment of the imaging apparatusassembly (101) with the orientation of the room (109);

6. identify an approximate mounting height (123) of the imagingapparatus assembly (101) above the floor plane (127) of the room (109),(e.g., by instructing the user to step back until the user is fullyvisible and unobstructed then confirm the approximate height of theuser, where the mobile application running on the mobile device (117)can provide some visual feedback of thermal images of the user capturedby the imaging apparatus assembly (101));

7. optionally, configure and mount additional thermal camera assemblies(e.g., installed in adjacent or opposite corners and/or edges, forexample, for improved fall detection where at least two thermal cameraassemblies are mounted in adjacent corners or edges), in a way similarto the configuring of the first imaging apparatus assembly (101) in theroom (109) without the needs to perform operations to identify theirmounting heights, because mounting height of the subsequently addedthermal camera assemblies can be computed from the height of an objectcaptured simultaneously by the first imaging apparatus assembly (101)and the subsequently added thermal camera assembly and the real worldheight calculated according to the mounting height of the first imagingapparatus assembly (101), and machine learning can be applied tocorrelate the objects as seen by the different thermal camera assembliesinstalled to monitor the same room (109)); and

8. optionally, go to one or more points of interests to identifyenvironmental features in the room (109) that may not be detectable fromthe thermal images of the room (109), such as the opposite corner of theroom (109) to define the maximum diagonal distance of the viewing field,other corners of the room (109) to help determine the ground plane ofthe scenery, and the locations of doors, tables, pillars, furniture, andother objects.

For example, the mobile application may instruct the user to walk to alocation and then push a button on the mobile application (or provide avoice commend to the mobile application, provide a gesture made viaflipping or shaking the mobile device (117), or stand still at thelocation for a few seconds) to indicate that the user is standing at thelocation. This teaches the thermal imaging system the geometry/layout ofthe room (109) and the locations and/or sizes of the environmentalobjects in the room (109) within the thermal image coordinate system.

For example, if a room has multiple doors, the installer could simplystand in the door, confirm in the mobile application that he/she isstanding in a door and chose from a set of menu options where this doorleads to (e.g., closet, kitchen, entrance, toilet, living room, etc.).Such information helps the thermal imaging system to determine peopletraffic (e.g., for monitoring a store or office spaces to determinewhere people movements are, for safety and/or security applications,etc.). Such an aspect is unique for thermal imaging because the POIsthat are invisible in the thermal infrared band are identified by simpleuser interaction with a mobile application running in the mobile device(117).

The storage of the input parameters, calibration parameters, POI mappingdata, etc., can be in the cloud (e.g., the server (113)), the mobiledevice (117), and/or the imaging apparatus assembly (101). The server(113) and/or the mobile device (117) can be configured to use algorithmsand/or look-up tables to reconstruct the geometrical relations betweenthe thermal image coordinate system and the room coordinate system,based on the configuration parameters.

When configured as discussed above, a thermal imaging system knows thetrue geometric relations between the thermal imaging space and the realworld space and/or the points of interest that are not visible in thethermal image system. As a result, the system can not only determineposition and height of humans and objects within the field of view ofthe thermal camera assemblies (e.g., 101)) but can also determine howmuch people have moved within the scenery.

For example, by tracking individuals throughout the scenery, the systemcalculates how much distance the individual has walked. Such informationcan be used in determining whether a senior has been sufficiently activewithin the premises for senior living; and a cloud based system cancalculate how much energy the subject has consumed/burned and aid theuser by interacting with the user and letting the user know if activityis insufficient. For example, if the activity is excessive and the riskof a fall/injury due to exhaustion is probable, the system can providean alert to the senior being monitored and/or care providers.

The configuration process discussed above can be carried out withminimum user requirements: the installer does not have to have anyskills in the related arts, does not require any instruments forinstallation; and for the parameter determination only an interfacedevice (e.g. a smart phone, tablet, or computer with an application orother pre-installed communication port) and a user height input areused.

In one aspect, the present application provides an imaging system whichcan determine automatically all configuration parameter in order todetermine geometric relations of the viewing field through distancemeasurement sensors or multiple (known) camera configuration. If suchengineering solution is not available (e.g. due to cost), a “simpler”imaging system determines the finial configuration parameter (e.g.,mounting height) based on an input from the user either the mountingheight or the height of the user/installer/the person performing thecalibration.

In another aspect, the present application provides a method where ahuman is a reference (marker) for providing novel functional contextualgeometric information and key points (e.g., points of interest) within aviewing field of an internet-connected thermal imaging apparatus whichhas the capability of computation and storage (e.g., via cloudcomputing) (and interact with user through a user interface) under a setof instruction without requiring the user/installer/human to havetechnical skills.

In a further aspect, the present application provides a cloud-basedthermal imaging system which reconstructs and provides contextualinformation of scenery by simple configuration determination.

The wirelessly connected thermal imaging system can use a low resolutionthermal footage to monitor human occupancy and activity in scenerywithout revealing information about person's identity. From the thermalfootage, the system determines human activity and well-being.

Optionally, the imaging apparatus assembly (101) contains an audiodevice to provide audio signals to occupants in the monitored area(e.g., room (109)). The audio signals may include voice instructionsand/or prompts streamed from the server (113). In such an embodiment,the user interface described above in connection with the mobile device(117) can be replaced or augmented with the audio-based interface. Forexample, the calibration/installation instructions provided by themobile device (117) can be replaced and/or augmented with voiceinstructions streamed from the server (113). For example, when the user(131) is at the location where the thermal image (133) of the user (131)is fully captured by a frame of image generated by the imaging apparatusassembly (101), a voice prompt instructs the user to remain standing atthe location while the system announces heights and start to walk whenan announced height matches the height of the user (131). Thus, the userheight can be conveniently provided to the system via a combination ofvoice prompts and thermal image feedback by motion or the lack of motion(and/or other gesture) that can be detected from the images generated bythe imaging apparatus assembly (101).

Optionally, the imaging apparatus assembly (101) includes a microphoneto receive audio inputs from the user(s) in the area monitored by theimaging apparatus assembly (101). Further, the imaging apparatusassembly (101) may include light-based indicators for user interaction.In some instances, the imaging apparatus assembly (101) uses acommunication connection to a separate device that has the audio and/orvisual capabilities to use the audio and/or visual capabilities toprovide the user interface. For example, an existing voice-basedintelligent personal assistant (e.g., in the form of aninternet-connected mobile device (117)) may be installed in the room(109) and connected to the access point (111); and the imaging apparatusassembly (101) connects to the personal assistant via the access point(111) to provide the interface for voice-based interaction; and thus itis not necessary to provide the user interface via a tablet computer(117).

The audio device(s) and/or visual device(s) of the imaging apparatusassembly (101) can be used to provide various services. For example, inthe case of elderly monitoring, when elderly do not hear properly, thatsome light indication can be triggered only based on human presence.

The imaging apparatus assembly (101) may be connected to other connecteddevices and/or systems, such as a landline telephone. For example, whena telephone is placed in a living room and it rings, the signal would besent via cloud (e.g., the server (113)) to the imaging apparatusassembly (101), which knows where the user (131) is present in the housefrom thermal imaging and thus provides audio and/or visual indicationsabout the event to the user (131) (e.g., light, beep, voice prompt,etc.). For example, the imaging apparatus assembly (101) may provideindications about an event from the from alarm burglar system. Fromtoddler monitoring to elderly care, the advantage of the system includesits non-intrusive people presence knowledge and non-wearable, remotenotification of events. The system can provide an attractive solutionwhere grandma is not required to wear something/carry something on her,in order to get notified about some events occurring in the house. Thesystem provides the notification based on the knowledge of the locationof grandma and/or the activity of the grandma. The notification can befiltered and/or provided in an intelligent way based on human activitiesobserved by the imaging apparatus assembly (101). For example, when thethermal image of grandma consistent with the grandma sleeping orwatching TV, certain alarms or notifications are suppressed.

For example, the imaging system includes a camera assembly (101) having:an enclosure (167) having at least two mounting surfaces (e.g., 162,163, and/or 164) that are orthogonal to each other for alignment with atleast two orthogonal surfaces (e.g., 171, 172, and/or 173) against whichthe camera assembly is to be mounted; at least one imaging apparatus(e.g., an imaging apparatus (175)) disposed within the enclosure (167)and having a predetermined orientation with respect to the enclosure(167); and a communication device disposed within the enclosure (167).The imaging system further includes a server (113) disposed at alocation remote from where the camera assembly (101) is mounted. Thecamera assembly (101) and the server (113) communicate over a computercommunication network (115) to identify at least one installationmeasurement of the camera assembly to establish a mapping from an imagecoordinate system for images generated by the imaging apparatus and areal world coordinate system aligned with an orientation defined by theat least two orthogonal surfaces. The communication device may be awireless communication device, or a wired communication device.

For example, a user (e.g., installer or owner) of the camera assembly(101) is instructed to mount the camera assembly (101) on a verticaledge (119) where two walls (171 and 172) meet; and the at least oneinstallation measurement includes a mounting height (123) of the cameraassembly (101) over a floor plane (127) on which the user (101) of thesystem stands.

Preferably but not required, the imaging apparatus is an imagingapparatus (175) that generates the images based on sensing infraredradiation. Preferably, a resolution of the imaging apparatus (175) issufficiently low that an identity of the person captured in the thermalimage generated by imaging apparatus (175) cannot be determined from thethermal image.

Optionally, a mobile application running in a mobile device (117) isconfigured to provide a user interface (e.g., as illustrated in FIG. 13), in communication with at least one of: the camera assembly (101) andthe server (113), in identifying the installation measurement, such asthe mounting height (123) and/or the locations of points of interest inthe image coordinate system (139).

For example, the user interface is configured to receive an inputidentifying a height of the user (131) whose thermal image (133) iscaptured in an image generated by the imaging apparatus (175); and themounting height (123) is computed from a height of the thermal image(133) of the user in the image coordinate system (139) and thereal-world height of the user (131) received in the user interface.

The at least one installation measurement may include a location, in theimage coordinate system, of a point of interest (e.g., a room corner, adoor, or a window) in a scene, area, or space monitored by the imagingapparatus (175). The point of interest is typically within the imagesgenerated by the imaging apparatus (175) but not visible in the imagesgenerated by the thermal camera. The installation measurement can beused to construct an area layout that defines the geometry of themonitored space.

In some implementations, the imaging system includes: a second cameraassembly having a known mounting height (e.g., previously determined,measured automatically using a sensor, or identifying by a user). Then,the mounting height of a first camera assembly can be computed based onthe mounting height of the second camera assembly and correlation ofobjects simultaneously captured in images generated the first and secondcameras. The mounting height of the first camera assembly can beadjusted such that the real world heights of objects observed andcalculated by the first camera match with the real world heights ofcorresponding objects observed and calculated by the second camera.

In some instances, a camera assembly (101) includes a sensor toautomatically measure a mounting height (123) between the cameraassembly (101) and a floor plane (127).

For example, after attaching the camera assembly (101) to an edge (119)or corner (174) where two or three orthogonal surfaces (e.g., 171, 172,and/or 173), the user may activate the camera assembly (101) toestablish a communication connection with a remote server (113) and/or amobile device (117). The server (113) and/or the mobile device (117) canprovide instructions the user to move around in the monitored area sothat the user is in a location where the full height of the thermalimage (133) of the user is detected in the image generated by the cameraassembly (101). The user may be prompted to provide a height of the userstanding on a floor (127) and captured in full by the camera assembly sothat the imaging system can compute a mounting height (123) of thecamera assembly (101), based on the real world height of the user (131)and a measurement of a height of the user in the image. The height ofthe user may be provided via a graphical user interface of the mobiledevice (117), or a gesture of the user detected via the camera assembly(101) in connection with voice prompts provided by the server (113).

Optionally, the user is instructed to move a thermally-detectable object(e.g., a cup of hot or cold water, or the body of the user) to a pointof interest in the area (e.g., room (109)) monitored by the cameraassembly (101) to allow the imaging system to bookmark the point ofinterest in images generated by the camera assembly (101) according to alocation of a thermal image (133) of the object positioned at, or in thevicinity of, the point of interest in the monitored area of the imagingsystem. For example, the camera assembly (101) images based on sensinginfrared radiation; and the point of interest is not visible in theimages generated by the camera assembly and thus cannot be determineddirectly from an analysis of the images generated by the camera assemblyat the time of installation.

Optionally, the imaging system identifies the locations of some pointsof interest from machine learning of objects identified from the thermalimages over a period of time, where temperature changes in certain areasof the monitored area and/or the human activities (and/or other thermalactivities) in the monitored area provide indications of the locationsof the points of interests. In such an implementation, it is notnecessary to provide a user interface for the calibration, calculation,and/or the identification of configuration parameters, such as themounting height, the location of points of interests, etc. For example,the user may attach the imaging apparatus assembly (101) and walks away;and the imaging system captures height of reference object andapproximates after time possible height range statistically. Such theinteraction of the imaging system with a user is optional; and thesystem performs the calibration in the background based on statisticalanalysis and/or “machine learning” of object identification from theresult of a large number units of camera assemblies installed in variouslocations and settings. Statistical results of objects and/orenvironments as observed by the camera assemblies and/or look up tablescan be used to train the imaging system to automatically calculate themounting heights and points of interests as recognized from the recordedimages from the camera assemblies.

In some instances, user inputs are provided to the imaging system viacorrelation of a known context (e.g., a user is instructed to go to apoint of interest) and the thermal images of the objects observed by theimaging apparatus assembly (101) (e.g., the location of a thermal image(133) of the user (131)). User inputs can also be provided through themobile device (117) by pushing a button in a user interface, a voicecommand to a user interface implemented on the mobile device (117), or agesture input using the mobile device (117). Inputs can also be made viathermal gesture detectable by the imaging apparatus assembly (101),e.g., by moving the object and then keeping the object still for atleast a predetermined period of time.

Examples of the points of interest include: a corner of a room in whichthe camera assembly is installed; a door of the room; a window of theroom; a furniture located in the room; a pathway in the room; and anactivity area in the room.

After the thermal image system is calibrated or configured with a set ofconfiguration parameters to map between: an image coordinate system ofimages generated by the camera assembly (101); and a real worldcoordinate system of the area monitored by the camera assembly (101),the thermal image system can provide valuable services.

For example, the thermal imaging system identifies sizes andorientations of objects visible in the images generated by the camera,based on sizes and orientation of the objects as measured in the imagesgenerated by the camera and the set of configuration parameters.

For example, the thermal imaging system generates monitoring alertsprovided via an output device of the camera assembly in reference to thepoints of interest in the area, when the set of configuration parametersfurther identifies points of interest in the area in the imagecoordinate system (e.g., points of interest having locations in theimages generated by the camera assembly but not visible in such images).

In some instances, the thermal image system can improve the set ofconfiguration parameters through statistical analysis and/or machinelearning. For example, the accuracy of the mounting height (123) of thecamera assembly (101) above a floor plane (127) of the monitored areacan be improved based on matching a statistical distribution of heightsof thermal images of humans observed by the imaging apparatus assembly(101) over a period of time with a known distribution.

Further calibration techniques of a thermal imaging system can be foundin U.S. patent application Ser. No. 15/607,345, filed May 26, 2017 andentitled “Apparatus and Method of Location Determination in a ThermalImaging System”, the disclosure of which is hereby incorporated hereinby reference.

Each of the mobile device (117), the server system (113), and theimaging apparatus assembly (101) can be implemented at least in part inthe form of one or more data processing systems illustrated in FIG. 14 ,with more or fewer components.

The present disclosure includes the methods discussed above, computingapparatuses configured to perform methods, and computer storage mediastoring instructions which when executed on the computing apparatusescauses the computing apparatuses to perform the methods.

FIG. 14 shows a data processing system that can be used to implementsome components of embodiments of the present application. While FIG. 14illustrates various components of a computer system, it is not intendedto represent any particular architecture or manner of interconnectingthe components. Other systems that have fewer or more components thanthose shown in FIG. 14 can also be used.

In FIG. 14 , the data processing system (200) includes an inter-connect(201) (e.g., bus and system core logic), which interconnects amicroprocessor(s) (203) and memory (211). The microprocessor (203) iscoupled to cache memory (209) in the example of FIG. 14 .

In FIG. 14 , the inter-connect (201) interconnects the microprocessor(s)(203) and the memory (211) together and also interconnects them toinput/output (I/O) device(s) (205) via I/O controller(s) (207). I/Odevices (205) may include a display device and/or peripheral devices,such as mice, keyboards, modems, network interfaces, printers, scanners,video cameras and other devices known in the art. When the dataprocessing system is a server system, some of the I/O devices (205),such as printers, scanners, mice, and/or keyboards, are optional.

The inter-connect (201) includes one or more buses connected to oneanother through various bridges, controllers and/or adapters. Forexample, the I/O controllers (207) include a USB (Universal Serial Bus)adapter for controlling USB peripherals, and/or an IEEE-1394 bus adapterfor controlling IEEE-1394 peripherals.

The memory (211) includes one or more of: ROM (Read Only Memory),volatile RAM (Random Access Memory), and non-volatile memory, such ashard drive, flash memory, etc.

Volatile RAM is typically implemented as dynamic RAM (DRAM) whichrequires power continually in order to refresh or maintain the data inthe memory. Non-volatile memory is typically a magnetic hard drive, amagnetic optical drive, an optical drive (e.g., a DVD RAM), or othertype of memory system which maintains data even after power is removedfrom the system. The non-volatile memory may also be a random accessmemory.

The non-volatile memory can be a local device coupled directly to therest of the components in the data processing system. A non-volatilememory that is remote from the system, such as a network storage devicecoupled to the data processing system through a network interface suchas a modem or Ethernet interface, can also be used.

In this description, some functions and operations are described asbeing performed by or caused by software code to simplify description.However, such expressions are also used to specify that the functionsresult from execution of the code/instructions by a processor, such as amicroprocessor.

Alternatively, or in combination, the functions and operations asdescribed here can be implemented using special purpose circuitry, withor without software instructions, such as using Application-SpecificIntegrated Circuit (ASIC) or Field-Programmable Gate Array (FPGA).Embodiments can be implemented using hardwired circuitry withoutsoftware instructions, or in combination with software instructions.Thus, the techniques are limited neither to any specific combination ofhardware circuitry and software, nor to any particular source for theinstructions executed by the data processing system.

While one embodiment can be implemented in fully functioning computersand computer systems, various embodiments are capable of beingdistributed as a computing product in a variety of forms and are capableof being applied regardless of the particular type of machine orcomputer-readable media used to actually effect the distribution.

At least some aspects disclosed can be embodied, at least in part, insoftware. That is, the techniques may be carried out in a computersystem or other data processing system in response to its processor,such as a microprocessor, executing sequences of instructions containedin a memory, such as ROM, volatile RAM, non-volatile memory, cache or aremote storage device.

Routines executed to implement the embodiments may be implemented aspart of an operating system or a specific application, component,program, object, module or sequence of instructions referred to as“computer programs.” The computer programs typically include one or moreinstructions set at various times in various memory and storage devicesin a computer, and that, when read and executed by one or moreprocessors in a computer, cause the computer to perform operationsnecessary to execute elements involving the various aspects.

A machine readable medium can be used to store software and data whichwhen executed by a data processing system causes the system to performvarious methods. The executable software and data may be stored invarious places including for example ROM, volatile RAM, non-volatilememory and/or cache. Portions of this software and/or data may be storedin any one of these storage devices. Further, the data and instructionscan be obtained from centralized servers or peer to peer networks.Different portions of the data and instructions can be obtained fromdifferent centralized servers and/or peer to peer networks at differenttimes and in different communication sessions or in a same communicationsession. The data and instructions can be obtained in entirety prior tothe execution of the applications. Alternatively, portions of the dataand instructions can be obtained dynamically, just in time, when neededfor execution. Thus, it is not required that the data and instructionsbe on a machine readable medium in entirety at a particular instance oftime.

Examples of computer-readable media include but are not limited torecordable and non-recordable type media such as volatile andnon-volatile memory devices, read only memory (ROM), random accessmemory (RAM), flash memory devices, floppy and other removable disks,magnetic disk storage media, optical storage media (e.g., Compact DiskRead-Only Memory (CD ROM), Digital Versatile Disks (DVDs), etc.), amongothers. The computer-readable media may store the instructions.

The instructions may also be embodied in digital and analogcommunication links for electrical, optical, acoustical or other formsof propagated signals, such as carrier waves, infrared signals, digitalsignals, etc. However, propagated signals, such as carrier waves,infrared signals, digital signals, etc. are not tangible machinereadable medium and are not configured to store instructions.

In general, a machine readable medium includes any mechanism thatprovides (i.e., stores and/or transmits) information in a formaccessible by a machine (e.g., a computer, network device, personaldigital assistant, manufacturing tool, any device with a set of one ormore processors, etc.).

In various embodiments, hardwired circuitry may be used in combinationwith software instructions to implement the techniques. Thus, thetechniques are neither limited to any specific combination of hardwarecircuitry and software nor to any particular source for the instructionsexecuted by the data processing system.

Other Aspects

The description and drawings are illustrative and are not to beconstrued as limiting. The present disclosure is illustrative ofinventive features to enable a person skilled in the art to make and usethe techniques. Various features, as described herein, should be used incompliance with all current and future rules, laws and regulationsrelated to privacy, security, permission, consent, authorization, andothers. Numerous specific details are described to provide a thoroughunderstanding. However, in certain instances, well known or conventionaldetails are not described in order to avoid obscuring the description.References to one or an embodiment in the present disclosure are notnecessarily references to the same embodiment; and, such references meanat least one.

The use of headings herein is merely provided for ease of reference, andshall not be interpreted in any way to limit this disclosure or thefollowing claims.

Reference to “one embodiment” or “an embodiment” means that a particularfeature, structure, or characteristic described in connection with theembodiment is included in at least one embodiment of the disclosure. Theappearances of the phrase “in one embodiment” in various places in thespecification are not necessarily all referring to the same embodiment,and are not necessarily all referring to separate or alternativeembodiments mutually exclusive of other embodiments. Moreover, variousfeatures are described which may be exhibited by one embodiment and notby others. Similarly, various requirements are described which may berequirements for one embodiment but not other embodiments. Unlessexcluded by explicit description and/or apparent incompatibility, anycombination of various features described in this description is alsoincluded here. For example, the features described above in connectionwith “in one embodiment” or “in some embodiments” can be all optionallyincluded in one implementation, except where the dependency of certainfeatures on other features, as apparent from the description, may limitthe options of excluding selected features from the implementation, andincompatibility of certain features with other features, as apparentfrom the description, may limit the options of including selectedfeatures together in the implementation.

In the foregoing specification, the disclosure has been described withreference to specific exemplary embodiments thereof. It will be evidentthat various modifications may be made thereto without departing fromthe broader spirit and scope as set forth in the following claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative sense rather than a restrictive sense.

What is claimed is:
 1. An apparatus comprising: a processor; a display;a user interface; and a memory connected to the processor, wherein thememory comprises instructions stored thereon configured to cause theprocessor to: receive, from an imaging apparatus, data indicative of athermal image captured by the imaging apparatus; output, to the display,the data indicative of the thermal image along with a prompt related toan object in the thermal image; and receive, via the user interface, aninput from a user, wherein the input is responsive to the prompt.
 2. Theapparatus of claim 1, wherein the prompt comprises a question requestingan actual height of the object in the thermal image.
 3. The apparatus ofclaim 2, wherein the object is a person.
 4. The apparatus of claim 1,wherein the input is a first input, and further wherein the instructionsfurther cause the processor to receive, via the user interface, a secondinput from the user, wherein the second input is indicative of aselection of the object in the thermal image.
 5. The apparatus of claim4, wherein the object is one of a plurality of objects depicted in thethermal image.
 6. The apparatus of claim 1, wherein the instructionsfurther cause the processor to transmit the input to a server computingdevice at least in part via a wireless communication network.
 7. Theapparatus of claim 6, wherein the data is first data, and wherein theinstructions further cause the processor to: receive, from the servercomputing device, second data indicative of a mounting heightinstruction for mounting the imaging device; and output, to the display,the mounting height instruction for mounting the imaging device, whereinthe mounting height instruction comprises a height at which the imagingdevice should be mounted.
 8. The apparatus of claim 7, wherein themounting height instruction is determined by the server computing devicebased at least on part on the input responsive to the prompt.
 9. Theapparatus of claim 1, wherein the instructions further cause theprocessor to: determine, by the processor based at least on part on theinput responsive to the prompt, a mounting height instruction formounting the imaging device; and output, to the display, the mountingheight instruction for mounting the imaging device, wherein the mountingheight instruction comprises a height at which the imaging device shouldbe mounted.
 10. The apparatus of claim 1, wherein the instructionsfurther cause the processor to output, to the display, instructions formoving the object to a specific location within a field of view of theimaging apparatus.
 11. The apparatus of claim 1, wherein the imagingapparatus comprises a thermal imaging apparatus.
 12. The apparatus ofclaim 1, wherein the object is a first object, and wherein theinstructions further cause the processor to determine an inferred heightof a second object based at least in part on the input responsive to theprompt.
 13. An apparatus comprising: an enclosure having at least oneface transparent to infrared radiation and at least one mounting faceconfigured to attach to a wall or ceiling of a room; a thermal imagingapparatus within the enclosure having a field of view oriented at leastin part to coincide with the at least one face transparent to infraredradiation; a processor; and a memory connected to the processor, whereinthe memory comprises instructions stored thereon configured to cause theprocessor to: receive data indicative of one or more thermal imagescaptured by the thermal imaging apparatus; determine a plurality ofobjects in the one or more thermal images; determine a statisticaldistribution of heights of the plurality of objects in the one or morethermal images; compare the statistical distribution with a knowndistribution of heights of other objects; and determine a mountingheight for the apparatus based at least in part on the comparison of thestatistical distribution of the heights of the plurality of objects inthe one or more thermal images with the known distribution of theheights of other objects.
 14. The apparatus of claim 13, wherein theplurality of objects are different humans.
 15. The apparatus of claim13, further comprising a wireless transmitter, and further wherein theinstructions further cause the processor to transmit, via the wirelesstransmitter, the mounting height to a computing device separate from theapparatus.
 16. An apparatus comprising: an enclosure having at least oneface transparent to infrared radiation; a thermal imaging apparatuswithin the enclosure having a field of view oriented at least in parttoward the at least one face; a processor; and a memory connected to theprocessor, wherein the memory comprises instructions stored thereonconfigured to cause the processor to: determine a plurality of objectsin one or more thermal images captured by the thermal imaging apparatus;determine a height of at least one object in the one or more thermalimages; and determine a mounting height for the apparatus based at leastin part on the height of the at least one object.
 17. The apparatus ofclaim 16, wherein the at least one object comprises a door opening area.18. The apparatus of claim 16, wherein the at least one object comprisesa human.
 19. The apparatus of claim 16, wherein the at least one objectcomprises a human standing in a door opening area.
 20. The apparatus ofclaim 16, wherein the at least one object comprises a plurality ofhumans.